This disclosure concerns a method for reconstructing the three-dimensional surfaces of opaque solid objects from a plurality of two-dimensional, preferably binary, images for automated determination of shape, location and orientation of the object.
In the science of robotics and computer vision, it is frequently important to determine the shape, orientation and location of a solid object. For example, when a robot gripper is to retrieve an object from a bin containing a number of the same objects, such a determination must be made. The determination is necessary in order that a robot gripper can be oriented and positioned to select an object from the top of the bin. Otherwise, no object may be retrieved or the parts and/or gripper may be damaged. In general, automated instrumentation is able to create two-dimensional images of three-dimensional objects. By using information gleaned from one or more of these two-dimensional images, decisions on object shape, location and orientation are made, usually by a computer. However, many of these known techniques require a priori knowledge of the object shape to simplify the orientation and location decisions.
In some of the known techniques, two similar two-dimensional views of the same scene are created. These two views are matched in some fashion to produce three-dimensional information for solving the object location problem. It is well known that the two pictures may produce ambiguities when combined. This ambiguity is called the correspondence problem. Many of these techniques use sophisticated mathematical manipulation of the digitized information that comprises the two-dimensional images in order to derive third dimension information. Substantial amounts of computing power are consumed in calculating the three-dimensional information that can be obtained from the two views.
In a different technique, one or more images of the same scene, for example parts disposed in a bin, are made from a fixed point but under varying illumination conditions. Surface orientations are then determined from the relative brightness of various points in each of the images. This method suffers from an inability to determine three-dimensional information concerning areas of the images that are always in shadows.
Range finding techniques employing beams of ultrasonic energy with time of flight estimating circuitry have also been used to calculate third dimension information . This method is inaccurate because the range finding beams are wide resulting in substantial errors, especially in gathering data about relatively small objects.
Coherent light may be used to narrow the beam width, but times of flight are difficult to measure at the speed of light. Using laser illumination in the known techniques is particularly time consuming since surfaces are illuminated and reconstructed only one point at a time.
In another method involving illumination, alternating light and dark stripes are projected on objects. However, the pattern produced frequently includes discontinuities that interfere with reconstruction of the third dimension information. Likewise, light from a source including a grid of intersecting pairs of parallel lines can be projected on a scene. However, third dimension information derived from the shape of the projected lines versus the initial grid can result in substantial errors in shape determination.
The described and other techniques that measure the intensity of light reflected objects in a scene suffer from a number of common problems. All depend, to some degree, on the smoothness or texture of the surface. A shiny surface producing "glint " usually significantly interferes with the efficiency of these methods. In addition, absent a priori knowledge of object shape, the methods are not robust. For example, variations from presumed surface reflection characteristics can result in substantial errors in third dimensional data. Each of the known light reflection methods suffers from other similar sensitivities, that is, variations in the conditions under which two-dimensional data is gathered can result in substantial error in the calculation of the third dimension information. Moreover, many of the known methods can only reconstruct three-dimensional information concerning objects of relatively simple shapes.
Accordingly, in order to reconstruct the third dimension of objects reliably from two-dimensional data, a method is needed that is not dependent on a priori knowledge of object shape, that does not consume significant computational power, that is not dependent on the type of surface finish of the object being reconstructed, that can reconstruct nearly any type of object that is opaque to a sensing beam and that is relatively insensitive to variations in the conditions under which the two-dimensional information is gathered.